CLAIMS 

What is claimed is: 

1. A computer-implemented method for a product shopping server 
system to automatically select product reviews to present to product shoppers, the 
selection of product reviews based at least in part on reputations of evaluators of 
the product reviews, the method comprising: 

repeatedly analyzing usefulness of product reviews to shoppers by, 

receiving a textual review of a product available from the product 
shopping server that is authored by a user acting as a product reviewer; 

receiving evaluations of the product review from each of multiple 
other users acting as review evaluators, each evaluation including a numerical 
rating of the product review for each of multiple predefined rating dimensions, 
each rating dimension related to an aspect of the product review such that a 
numerical rating for the rating dimension indicates an assessment by the review 
evaluator of a degree to which that aspect of the product review is satisfied; 

automatically determining when an evaluator reputation calculation 
threshold is reached for the product review, the determining based at least in part 
on a quantity of the received evaluations for the product review; 

after the evaluator reputation calculation threshold is reached for the 
product review, automatically calculating an updated evaluator reputation weight 
for each of the review evaluators based on the received evaluations for the 
product review, the evaluator reputation weight of a review evaluator reflecting an 
ability of that review evaluator to generate accurate evaluations and based at least 
in part on a comparison of evaluations previously received from that review 
evaluator to evaluations previously received from other review evaluators; and 

automatically generating a weighted average rating of the product 
review for each of the multiple rating dimensions by averaging the ratings received 
for a rating dimension in such a manner that a rating from a review evaluator is 
weighted using the evaluator reputation weight for that review evaluator; and 
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when a shopper requests information regarding a product, selecting one or 
more of the product reviews received for the product to present to the shopper 
based on the automatically generated ratings for the selected product reviews, the 
selecting such that the selected product reviews have ratings that indicate a high 
degree of satisfaction for each of multiple of the aspects of those product reviews, 
so that shoppers receive product reviews that are useful based on automatically 
generated ratings that reflect reputations of evaluators of the product review. 

[c2] 2. The method of claim 1 wherein the determining that the evaluator 

reputation calculation threshold is reached for a product review further includes 
determining that a consensus was reached by the review evaluators for that 
product review as to the degrees of satisfaction of the aspects of that product 
review for the predefined rating dimensions. 

[c3] 3. The method of claim 1 wherein the generated weighted average 

rating for at least one of the rating dimensions of each of the product reviews is 
further weighted based on sales weights assigned to each of at least some of the 
review evaluators, the sales weight for a review evaluator reflecting an amount of 
prior sales to that review evaluator. 

[c4] 4. The method of claim 1 wherein the generated weighted average 

rating for at least one of the rating dimensions of each of the product reviews is 
further weighted based on an author reputation weight assigned to the product 
reviewer that authored the product review, the author reputation weight of a 
product reviewer reflecting an ability of that product reviewer to generate accurate 
product reviews. 

[c5] 5. The method of claim 4 wherein the ability of a product reviewer to 

generate accurate product reviews that is reflected in the author reputation weight 
of the product reviewer is based on a degree of consistency between weighted 
average ratings generated for product reviews previously authored by the product 
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reviewer for multiple distinct products, and wherein the ability of a review 
evaluator to generate accurate evaluations that is reflected in the evaluator 
reputation weight of the review evaluator is based on a degree of agreement 
between evaluations for multiple distinct product reviews that were previously 
received from the review evaluator and consensus evaluations for those product 
reviews from multiple other review evaluators. 

[c6] 6. The method of claim 4 including, for each product review authored 

by a product reviewer, after an author reputation calculation threshold is reached 
for the product review based on the evaluations received for the product review, 
automatically calculating an updated author reputation weight for the product 
reviewer based on the received evaluations for the product review. 

[c7] 7. The method of claim 4 including, after a shopper purchases a 

product, querying the shopper to obtain feedback regarding actual usefulness to 
the shopper of the product reviews selected for presentation to the shopper, and 
updating based on the shopper feedback the author reputation weight for the 
product reviewers that authored those product reviews and/or the evaluator 
reputation weights for the review evaluators that provided evaluations for those 
product reviews. 

[c8] 8. The method of claim 1 including ranking each product reviewer 

relative to other product reviewers based at least in part on automatically 
calculated author reputation scores for those product reviewers, ranking each 
review evaluator relative to other review evaluators based at least in part on 
automatically calculated evaluator reputation scores of those review evaluators. 
and providing visible feedback to users of the product shopping server of the 
product reviewers and the review evaluators that have high rankings. 

[c9] 9. The method of claim 1 including, for each product review authored 

by a product reviewer, delaying the automatic generating of the weighted average 

45 



ratings for the product review until after a content rating calculation threshold is 
reached for the product review based on the evaluations received for the product 
review. 



[ciO] 10. The method of claim 1 wherein at least some of the product reviews 

authored by product reviewers are blurbs obtained from blogs authored by those 
product reviewers. 

[cii] 11. A method in a computing device for selecting information to provide 

to users based on reputations of evaluators of the information, the method 
comprising: 

receiving from a reviewer user a review related to an available item; 

receiving evaluations of the review from each of multiple evaluator users, 
each received evaluation including a quantitative assessment of contents of the 
review for each of one or more of multiple content rating dimensions available for 
use in assessing the review, each of the evaluator users having an existing 
reputation weight based at least in part on previous evaluations; 

automatically generating at least one aggregate assessment of the content 
of the review based at least in part on combining quantitative assessments from 
the received evaluations for the review, at least one of the generated aggregate 
assessments being further based on the reputation weights of the evaluator users 
in such a manner that a first quantitative assessment from a first evaluator user 
with a first reputation weight has a different impact on that generated aggregate 
assessment than that first quantitative assessment from a distinct second 
evaluator user with a distinct second reputation weight; 

automatically updating the reputation weights for each of one or more of 
the evaluator users based on a relationship of the quantitative assessments from 
the evaluation of that evaluator user to the quantitative assessments from the 
evaluations of other of the evaluator users; and 
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determining whether to provide the review to another user based at least in 
part on one or more of the automatically generated aggregate assessments for 
the content of the review. 



[ci2] 12. The method of claim 11 including, before the automatic updating of 

the reputation weights of the one or more evaluator users, determining whether 
the received evaluations satisfy an evaluator reputation calculation threshold, and 
wherein the automatic updating of the reputation weights of the one or more 
evaluator users is performed only when it is determined that the received 
evaluations satisfy the evaluator reputation calculation threshold. 

[ci3] 13. The method of claim 12 wherein the evaluator reputation calculation 

threshold is based at least in part on a minimum degree of consensus existing 
among the received evaluations, and wherein the determining includes 
automatically calculating the existing degree of consensus among the received 
evaluations. 

[ci4] 14. The method of claim 11 wherein the relationship of the quantitative 

assessments from the evaluation of an evaluator user to the quantitative 
assessments from the evaluations of other of the evaluator users that is used 
when automatically updating the reputation weight for that evaluator user is based 
on a degree of agreement between the quantitative assessments from the 
evaluation of the evaluator user and quantitative assessments from a consensus 
evaluation for the received evaluations. 

[ci5] 15. The method of claim 11 wherein the reputation weights of the 

evaluator users that are used in the automatic generating of the aggregate 
assessments of the content of the review were automatically generated based on 
the previous evaluations by those evaluator users. 
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[ci6] 16. The method of claim 11 including, after the receiving of the 

evaluations from the evaluator users, for each of at least some of the evaluations 
receiving one or more ratings of the evaluation from users other than the evaluator 
user that provided the evaluation, and automatically modifying the reputation 
weights for evaluator users whose evaluations received ratings based at least in 
part on those ratings. 

[ci7] 1 7. The method of claim 1 1 including, after the automatic updating of the 

reputation weights of the one or more evaluator users, receiving an indication that 
the content is no longer in use for determining reputation weights of the evaluator 
users, and automatically updating the reputation weights for each of those 
evaluator users to remove influence based on the relationship of the quantitative 
assessments from the evaluation of that evaluator user to the quantitative 
assessments from the evaluations of other of the evaluator users. 

[ci8] 18. The method of claim 11 wherein the automatic generating of the 

aggregate assessments of the content of the review is further based in part on an 
existing reputation weight of the reviewer user from which the review was 
received. 

[ci9] 19. The method of claim 18 wherein the reputation weight of the 

reviewer user is based on a degree of consistency between one or more of the 
automatically generated aggregate assessments of the content of the review and 
automatically generated aggregate assessments of the content of previous 
reviews received from the reviewer user. 

[c20] 20. The method of claim 11 including automatically updating a 

reputation weight of the reviewer user from which the review was received based 
at least in part on one or more of the automatically generated aggregate 
assessments of the content of the review. 
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[c2i] 21. The method of claim 20 including, before the automatic updating of 

the reputation weight of the reviewer user, determining whether the received 
evaluations satisfy an author reputation calculation threshold, and wherein the 
automatic updating of the reputation weight of the reviewer user is performed only 
when it is determined that the received evaluations satisfy the author reputation 
calculation threshold. 

[c22] 22. The method of claim 11 including, before the automatic generating 

of the aggregate assessments of the content of the review, determining whether 
the received evaluations satisfy a content rating threshold, and wherein the 
automatic generating of the aggregate assessments of the content of the review is 
performed only when it is determined that the received evaluations satisfy the 
content rating threshold. 

[c23] 23. The method of claim 21 wherein the content rating threshold is 

based at least in part on a weighted number of the evaluations received for the 
review from the evaluator users that is based on the reputation weights of the 
evaluator users in such a manner that an evaluation from a first evaluator user 
with a first reputation weight has a different Impact on that weighted number of 
evaluations than an evaluation from a distinct second evaluator user with a distinct 
second reputation weight. 

[c24i 24. The method of claim 1 1 wherein each of the received evaluations 

include quantitative assessments of the contents of the review for each of the 
multiple available content rating dimensions. 

[c25] 25. The method of claim 24 including, before the receiving of the 

evaluations of the review, determining the multiple available content rating 
dimensions. 
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[c26] 26. The method of claim 11 including, before the receiving of the 

evaluations of the review, soliciting the evaluator users to provide evaluations of 
the review, the solicitations including indications of the multiple available content 
rating dimensions. 

[c27] 27. The method of claim 11 wherein the automatic generating of the 

aggregate assessments of the content of the review includes generating an 
aggregate assessment for each of the multiple available content rating 
dimensions. 

[c28i 28. The method of claim 27 including automatically generating an overall 

aggregate assessment of the review based at least in part on the automatically 
generated aggregate assessments of the content of the review. 

[c29] 29. The method of claim 11 wherein the automatically generated 

aggregate assessments of the content of the review are each further based on a 
sales weighting for each of one or more of the evaluator users, the sales weighting 
of an evaluator user reflecting an amount of prior sales to that evaluator user. 

[c30] 30. The method of claim 1 1 including, after the automatic updating of the 

reputation weights for the evaluator users, ranking each evaluator user relative to 
other evaluator users based at least in part on automatically generated evaluator 
reputation scores of those evaluator users. 

[c3i] 31 . The method of claim 30 wherein the reputation weight for each of 

the evaluator users is based on a combination of quantity and quality of 
evaluations provided by that evaluator user, and including automatically 
generating a distinct evaluator reputation rating score for each of the evaluator 
users based solely on the quality of the evaluations provided by that evaluator 
user, and wherein the evaluator reputation scores used for the ranking are the 
evaluator reputation rating scores. 
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[c32] 32. The method of claim 30 including providing visible feedback to 

users of the rankings of at least some of the evaluator users. 

Ic33] 33. The method of claim 1 1 wherein at least some of the evaluator users 

each have multiple existing reputation weights that correspond to previous 
evaluations by those evaluator users of content of different categories, and 
including, before the automatic updating of the reputation weights for the evaluator 
users, determining a category of the review, and wherein the automatic updating 
of the reputation weights of evaluator users that have multiple existing reputation 
weights is performed for an existing reputation weight of that evaluator user for the 
determined category. 

[c34] 34. The method of claim 1 1 wherein at least some of the evaluator users 

each have multiple existing reputation weights that correspond to different types of 
activities previously performed by those evaluator users, and wherein the 
automatic updating of the reputation weights of evaluator users that have multiple 
existing reputation weights is performed for an existing reputation weight of that 
evaluator user corresponding to prior review evaluation activities of that evaluator 
user. 

[c35] 35. The method of claim 1 1 including, after the automatic updating of the 

reputation weights for the evaluator users, providing indications of the reputation 
weights for one or more of those evaluator users to one or more third-party 
computing devices so that they can interact with those evaluator users based on 
those reputation weights. 

[c36] 36. The method of claim 11 wherein the method is performed by the 

computing device on behalf of another computing system with whom the evaluator 
users are interacting, the method performed as a service to the another computing 
system. 
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[c37] 37. The method of claim 1 1 including receiving from one or more third- 

party computing devices information related to the reputations of one or more of 
the evaluator users, the received information based on interactions of those 
evaluator users with those third-party computing devices, and automatically 
updating the reputation weights for each of those evaluator users based on the 
received information. 

[c38] 38. The method of claim 1 1 wherein the review is information obtained 

from a blog authored by the reviewer user. 

[c39] 39. A computer-readable medium whose contents cause a computing 

device to determine information to provide to users based on reputations of 
evaluators of the information, by performing a method comprising: 
receiving information from a first user; 

receiving quantitative assessments from each of multiple evaluators for 
each of one or more specified qualities of the received information; 

automatically generating at least one aggregate assessment of the 
received information based at least in part on the received quantitative 
assessments and on reputation weights assigned to the evaluators that provided 
those assessments; and 

determining to provide the received information to a second user based at 
least in part on the automatically generated aggregate assessments for the 
received information. 

[c40] 40. The computer-readable medium of claim 39 wherein the method 

further includes updating the reputation weights for the evaluators that provided 
the assessments of the received information based on a relationship of the 
assessments of those evaluator users to a consensus of the assessments. 

[c4i] 41. The computer-readable medium of claim 39 wherein the computer- 

readable medium is a memory of a computing device. 
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[c42] 42. The computer-readable medium of claim 39 wherein the computer- 

readable medium is a data transmission medium transmitting a generated data 
signal containing the contents. 

[c43] 43. The computer-readable medium of claim 39 wherein the contents 

are instructions that when executed cause the computing device to perform the 
method. 

[c44i 44. A computing device for assessing information based on reputations 

of evaluators of the information, comprising: 

a content rater component able to, for each of multiple pieces of content, 
generate an aggregate assessment of the piece of content based on assessments 
of the piece of content received from multiple evaluators that are combined in a 
weighted manner based on previously generated reputation weights of the 
evaluators, the assessments each for one or more of multiple rating dimensions; 

an evaluator reputation assessor component able to, for each of at least 
some of the multiple pieces of content, update the reputation weights for each of 
the evaluators of the piece of content based on a comparison of the assessment 
from that evaluator of the piece of content to the assessments from the other 
evaluators of the piece of content; and 

a content manager system able to, for each of the multiple pieces of 
content, determine how to manage the piece of content based at least in part on 
the generated aggregate assessment of the piece of content. 

[c45] 45. The computing device of claim 44 wherein the content rater 

component and the evaluator reputation assessor component are executing in 
memory of the computing device. 

[c46] 46. The computing device of claim 44 wherein the content rater 

component consists of means for generating an aggregate assessment of a piece 
of content based on reputation weights of evaluators that assess the piece of 
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content, wherein the evaluator reputation assessor component consists of means 
for generating a reputation weight of an evaluator based on assessments received 
from the evaluator and on assessments received from other evaluators, and 
wherein the content manager system consists of means for determining how to 
manage a piece of content based on the generated aggregate assessment of the 
piece of content. 

[CAT] 47. A computer-implemented method for assessing information based 

on reputations of evaluators of the information, the method comprising: 

receiving evaluations of content from each of multiple evaluators, each 
received evaluation including a quantitative assessment of the content for each of 
one or more of multiple content rating dimensions; 

determining a consensus evaluation of the content based on the received 
evaluations; 

calculating one or more reputation scores for each of the evaluators based 
at least in part on a degree of match between the received evaluation from the 
evaluator and the determined consensus evaluation; and 

for each of the evaluators, using at least one of the calculated reputations 
scores for the evaluator to weight other assessments of other content received 
from the evaluator, so that content with weighted assessments can be managed in 
a manner based on those weighted assessments. 

[c48] 48. The method of claim 47 further including calculating a weighted 

assessment of the content based on weighting the quantitative assessments of 
the received evaluations using calculated reputation scores of the evaluators that 
provided the received evaluations and based on a reputation rating of a generator 
of the content that is weighted using a calculated reputation score of the 
generator, and managing the content based on the calculated weighted 
assessment of the content. 
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